Speech analysis and therapy system and method

ABSTRACT

A system and method for analyzing a speech problem includes the steps of presenting to a student/user a symbol representative of a word and prompting the user to pronounce the word represented by the symbol into a microphone in signal communication with a processor. Next the therapist enters a phonetic representation of the user pronunciation into the processor. It is then automatically determined whether an error exists in the user pronunciation. If an error exists, the error is automatically categorized. The system and method may be adapted for presentation of a single word, a plurality of words having a predetermined feature desired to be tested, a pretest for screening for potential articulation disorders, and an analysis of connected speech with the use of a moving picture to elicit a narrative from the student.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to systems and methods foranalyzing and remediating speech pathologies, and, more particularly, tosuch systems and methods that are computer-based.

[0003] 2. Description of Related Art

[0004] Articulation and phonology disorders are the most common of thespeech and language disorders. The prevalence of this disorder is, atthe time of writing, approximately 10% of the school-age population. Inaddressing a perceived articulation issue in a student, speech/languagepathologists have in the past used an initial test based upon a seriesof cards. Each card contains a picture and a word, and the student isasked to pronounce the word associated with the card. The pathologistthen determines whether the student's pronunciation is “right” or“wrong.” It may be recognized that such a system can be cumbersome,owing to the cards' having to be placed in a desired order and sortedmanually.

[0005] An intervention system designed to automate this process, PictureGallery I, was presented by the owner of the current application. Inthis system pictures and/or words stored in a database could be sortedusing a desired criterion such as a particular phoneme and presented tothe student under software control for facilitating the acquisition orremediation of speech or language skills. No analysis or scoring isperformed; rather, the product is intended for use by one or morestudents, either alone or in concert with a pathologist/teacher.

[0006] A previously known method of diagnosing articulation or phonologydisorders included a “pencil and paper” test wherein a student is askedto speak a word. The therapist grades the word subjectively, based uponthe therapist's ear and the local standards.

[0007] Other systems known in the art that address speech/languageanalysis and therapy methodologies includes those of Neuhaus (U.S. Pat.No. 6,113,393), Parry et al. (U.S. Pat. No. 6,077,085), UCSF and Rutgers(U.S. Pat. Nos. 5,813,862 and 6,071,123), Neumeyer et al. (U.S. Pat. No.6,055,498), Jenkins et al. (U.S. Pat. Nos. 5,927,988 and 6,019,607),Siegel (U.S. Pat. No. 6,009,397), Beard et al. (U.S. Pat. No.5,857,173), Aaron et al. (U.S. Pat. No. 5,832,441), Russell et al. (U.S.Pat. Nos. 5,679,001 and 5,791,904), Rothenberg (U.S. Pat. No.5,717,828), Wen (U.S. Pat. No. 5,562,453), and Ezawa et al. (U.S. Pat.No. 4,969,194). Commercial software products in the field ofarticulation, phonology, or speech sound production includeSpeechViewer, Interactive System for Phonological Analysis, SpeechMaster, Visi-pitch, and Computerized Profiling. Commercial printproducts include the Goldman-Fristoe Test of Articulation (AmericanGuidance Service), Khan-Lewis Test of Phonology (American GuidanceService), Photo Articulation Test (Pro-Ed), and Fisher-Logeman Test ofArticulation (Pro-Ed).

SUMMARY OF THE INVENTION

[0008] It is therefore an object of the present invention to provide asystem and method for eliciting a desired sound from a user.

[0009] It is a further object to provide such a system and methodadapted to generate a report.

[0010] It is another object to provide a system and method for testing auser's articulation.

[0011] It is an additional object to provide such a system and methodthat is adapted to analyze a group of problematic sounds.

[0012] It is yet a further object to provide such a system and methodthat recommends a therapeutic program responsive to the analysis.

[0013] It is yet another object to provide such a system and method thatincludes a prescreening feature.

[0014] These and other objects are achieved by the present invention, afirst aspect of which comprises a method and system for providing speechtherapy. The method comprises the steps of selecting a problem speechsound and searching a database that comprises a plurality of records.Each record comprises a picture and a word associated with the word.

[0015] Next a set of records is automatically generated from theplurality of records. Each record contains a word specific to theproblem speech's sound. The set of records is next automaticallypresented to a user sequentially on a display device, and the user isprompted to pronounce the displayed word. Finally, the pronunciation ofeach word is scored.

[0016] The system of the first aspect of the present invention comprisesa processor, an input device in communication with the processor havingmeans for selecting a problem speech sound, and a display device incommunication with the processor. The database as described above isresident on the processor, as are software means. The software isadapted to automatically generate a set of records from the plurality ofrecords, with each record containing a word specific to the problemspeech sound. The software is also adapted to automatically present atleast a portion of each record in the set of records to a usersequentially on a display device; the set of records to a usersequentially on the display device and to prompt the user to pronouncethe displayed word. Finally, the software is adapted to receive via theinput device a score for the pronunciation of each word.

[0017] Another aspect of the present invention is a system and methodfor analyzing a speech problem by performing a test of articulation,phonology, and sound features that is administered and analyzed with theuse of an electronic processor. This method comprises the steps ofpresenting to a student/user a symbol representative of a word andprompting the user to pronounce the word represented by the symbol intoa microphone in signal communication with a processor. Next thetherapist enters a phonetic representation of the user pronunciationinto the processor. It is then automatically determined whether an errorexists in the user pronunciation. If an error exists, the error isautomatically categorized.

[0018] The system of the second aspect of the invention evaluates anarticulation disorder. The system comprises a processor and an outputdevice and an input device, each in signal communication with theprocessor.

[0019] Software installable on the processor is adapted to present onthe output device, typically a display device, although this is notintended as a limitation, a symbol representative of a word. Thesoftware then is adapted to prompt a user via the output device topronounce the word represented by the symbol and to receive from thetherapist via the input device a phonetic representation of the user'spronunciation. The software automatically determines whether an errorexists in the user pronunciation, and, if an error exists, automaticallycategorizes the error.

[0020] The system and method of this second feature of the invention maybe adapted for presentation of a single word, a plurality of wordshaving a predetermined feature desired to be tested, a pretest forscreening for potential articulation disorders, and an analysis ofconnected speech with the use of a moving picture to elicit a narrativefrom the student.

[0021] The features that characterize the invention, both as toorganization and method of operation, together with further objects andadvantages thereof, will be better understood from the followingdescription used in conjunction with the accompanying drawing. It is tobe expressly understood that the drawing is for the purpose ofillustration and description and is not intended as a definition of thelimits of the invention. These and other objects attained, andadvantages offered, by the present invention will become more fullyapparent as the description that now follows is read in conjunction withthe accompanying drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

[0022]FIGS. 1A,1B is a flow chart for an exemplary embodiment of thespeech therapy method of the invention.

[0023]FIG. 2 is a schematic diagram of the speech therapy system.

[0024]FIGS. 3A,3B is a flow chart for an exemplary embodiment of thespeech analysis method of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] A description of the preferred embodiments of the presentinvention will now be presented with reference to FIGS. 1A-3B.

[0026] A flow chart of an exemplary embodiment of the automated speechtherapy/intervention method is given in FIGS. 1A,1B, and a schematic ofthe system in FIG. 2. The system and method are also contemplated foruse in the acquisition of a language skill as well as in a remediationsetting. There are two versions of the system and method: In the“professional” version 10 of the invention (block 100), typically twopeople who will be referred to as “therapist” 11 and “student” 12 arepresent, although this is not intended as a limitation. This version iscontemplated for use in such settings 32 as a hospital, clinic,rehabilitation center, school, or private facility. In the “personal”version 40 of the invention, the “student” 12 may be working alone, orin the presence of a nonprofessional such as a parent. The therapist 11may be, for example, a speech therapist or a teacher; the student 12 maybe a user who is learning a second language or a school attendee who isbeing tested for, or who is already known to have, an articulationproblem or phonological disorder.

[0027] The method comprises the steps of providing access to anelectronic database that includes a plurality of records (block 101).Each record comprises a word, a picture representative of the word, anda recommended pronunciation of the word. In an alternate embodiment, therecord may also include a digitized video clip to represent motion or averb to impart a concept of action. In another embodiment the record mayfurther include a digitized sound that is associated with the word. Forexample, the record for the word dog might contain a picture of a dog, avideo clip of a dog running, and/or a barking sound. It is believed thatsuch multiple stimuli appeal to a multiplicity of cognitive areas,thereby optimizing the student's improvement.

[0028] Each record may further contain data useful for performingsorting functions, such as at least one category and/or concept. Anexemplary set of categories comprises: animals, art, babies,celebrations, global images, environment, family, food, garden, healthand exercise, home, leisure, medical, money, music, pets, play, school,shopping, signs/symbols, sports, technical, vacations, and work. Anexemplary set of concepts comprises: activities, objects, places,people, ideas, and events. The record also typically comprises avocabulary level associated with the word and a length of the word.

[0029] The method next comprises the step of inputting or accessingpreviously input demographic information for the student (block 102).Then a problem speech sound that is desired to be improved upon isselected that is known from a prior diagnosis (block 103). The problemspeech sound may be selected from a group consisting of a phoneme and a“feature.” The feature comprises at least one of a place, a manner, anda voicing characteristic. Searching on a feature yields matches in allpositions of words. The database is electronically searched (block 106)for records containing words that include the problem speech sound togenerate a set of records. A filter may be applied if desired (block104) to further limit the set (block 105), including selecting acategory or concept, using the demographic information to limit the set,such as eliminating words that are intended for students over 7 years ofage for a 5-year-old student, setting a desired vocabulary level, orselecting a word length.

[0030] If desired (block 107), the set of records may also be sorted(block 108) in various ways to produce a desired sequence, including,but not limited to, putting the words in alphabetical order, randomorder, or some other chosen sequence. In a preferred embodiment, all thewords in the database contain at least one of the letters “r,” “l,” and“s” since these are known to present a problem most frequently.

[0031] For a professional therapy session, a decision may be madewhether to present the set of records or store/transmit them (block109). If the former, the set of records is next presented sequentiallyto the student in the predetermined sequence on a display device (block111), and the student is prompted to pronounce the word (block 112). Thedisplay style may be selected (block 110) from a word only, a pictureonly, or a word plus a picture.

[0032] If the student can read, he or she can use the displayed word toform a pronunciation; if the student cannot yet read, or cannot read thecurrently presented language, the picture will also aid in acquisitionof reading skills as well as pronunciation.

[0033] In the professional setting, the therapist scores the student'spronunciation (block 113) by inputting, for example, “correct,”“incorrect,” “skip,” or “re-present,” which will record an indication tore-present the record at a later time, such as after all the other itemsin the set have been presented. The student or therapist can also elect(block 114) to hear the word pronounced (block 115) in a recommendedmanner by making an appropriate selection on an input device.

[0034] The scores are received by the system, and an aggregate score iscalculated (block 116) for the problem speech sound. The database alsocomprises a historical record of all sessions for each of the students,and the database is then accessed to store the current score thereinto(block 117). The therapist may choose to calculate a historical change(block 118) from previously saved scores to provide an indication of thestudent's progress. Such scores may also be used to calculate statistics(block 119) for a group of students, using, for example, a demographicfilter.

[0035] The “personal version” of the system and method does not acceptscoring, nor is there a database from which sets of records may becreated. Rather, the professional version is adapted to download aselected set of records onto a storage medium, such as a diskette, or totransmit the set of records to a remote site (block 109). Such a remotesite may comprise, but is not intended to be limited to, a room remotefrom the main processor accessible via intranet, or a different buildingaccessible via internet. This version then enables the student toperform (block 120) the steps in blocks 110-112 and 115 as desired onhis or her own.

[0036] The system 10, as schematically illustrated in FIG. 2, comprisesa processor 14, on which are resident the software package 15 of thepresent invention adapted to perform the functions as outlined above anda database 16 comprising the plurality of records 17 and demographic andhistorical data on the users 12. An input device is in communicationwith the processor 14 that has means for selecting a problem speechsound. Such means may comprise any of the devices known in the art suchas a keyboard 18 or pointing device such as a mouse 19 or touch screen.A display device such as a display screen 20 is also in communicationwith the processor 14.

[0037] Optional elements that are also in communication with theprocessor 14 may include a microphone 21 and a speaker 22, both underprocessor 14 control, as well as means for performing analog-to-digital23 and digital-to-analog 24 conversions. The system 10 also has meansfor transferring records from the database to a storage medium such as adisk drive 25, under control of the software 15, or to a remote sitesuch as another location 26 via a modem 27 over the internet 28 or suchas another room 29 at the same location via an intranet 30. A printer 31under processor control may also be provided for furnishing a hard copyof any portion of the session as desired.

[0038] A secondary system 40 for use of the personal version of theinvention at the remote location 26,29 comprises a processor 41, inputdevice 42 and display device 43 in communication with the processor 41,and either or both of a modem 44 for receiving a set of records and astorage device reader 45 for reading a stored set of records. Thesoftware package 46 for this version is adapted to read the records,present them to the student 12 sequentially, and prompt the student 12to pronounce the word associated with the record.

[0039] A flow chart of an exemplary embodiment of the automated speechtherapy/intervention method is given in FIGS. 3A-3. The schematic of thesystem is substantially the same as that in FIG. 2. The method comprisesthe steps of selecting the type of evaluation desired to be performed(block 501): screening, single word analysis, “deep” test, or connectedspeech analysis. The screening, or pre-evaluation, comprises the stepsof presenting to a user a symbol representative of a word (block 502)and prompting the user to pronounce the word represented by the symbolinto a microphone in signal communication with a processor (block 503).The symbol presentation may comprise, for example, a picture on adisplay screen, although this is not intended as a limitation. Thetherapist then enters a phonetic representation of the userpronunciation into the processor (block 504). Software installed uponthe processor then automatically determines whether an error exists inthe user pronunciation (block 506). The determination may additionallyinclude the application of a dialectical filter (block 505) that isadapted to discriminate between that which is deemed to be a true errorand a predetermined normal dialect word pronunciation. If an errorexists, the software automatically categorizes the error (block 507). Anerror may be, for example, a substitution, a mispronunciation, or anomission. These steps are repeated a predetermined number of times n,for example, 20 times (block 510).

[0040] It may then be desired to perform the “deep test,” which may beperformed with the knowledge gained from a pre-evaluation as above or denovo. If the pre-evaluation has been performed, the softwareautomatically generates a set of symbols, wherein each symbol isrepresentative of a word containing at least one of the errorsdetermined in the pre-evaluation. Then the steps as above are performedusing the generated set of symbols, and an evaluation is made ofarticulation errors for the whole set.

[0041] If a single word is desired to be analyzed for, the steps inblocks 502-509 are performed once for the desired word.

[0042] Once a word has been pronounced and the phonetic representationentered into the processor, the therapist may decide to display afrequency spectrum of the user's pronunciation (block 508). If desired,a sample of a correct pronunciation of the word may be broadcast via aspeaker in signal communication with the processor (block 509).

[0043] When a plurality of words have been tested, the evaluating stepalso comprises automatically recognizing an underlying commonality bycorrelating the errors detected. This pattern recognition permits thesoftware to achieve an overarching diagnosis of a problem speech sound(block 511).

[0044] Following the error categorization, if desired, a report can beissued detailing the user's error(s) (block 512). Additionally, theerror may be saved in a database that is accessible by the processor(block 513). If a previous entry for this user already exists, which isdetermined by a search, the error found in the present test may becompared with an error previously found, and a change over timedetermined for that user (block 514), to note whether an improvement hasoccurred. Again, if desired, a report may be issued (block 515) as tothe change determined.

[0045] An additional feature of this invention is the ability, once acategorization has been made of an error, of recommending a therapeuticprogram to address the error (block 516). Such a recommendationformulation may comprise, for example, creating a set of records asdetailed above in FIGS. 1A-2.

[0046] If connected speech analysis is desired to be performed, the“symbol” comprises a motion picture representative of an action, and theuser is prompted to provide a narration on the action into a microphonein signal communication with a processor. The therapist then enters aphonetic representation of the user's pronunciation of the narrationinto the processor. Software resident in the processor automaticallydetermines whether an error exists in the user pronunciation, and, if anerror exists, automatically categorizes the error.

[0047] It may be appreciated by one skilled in the art that additionalembodiments may be contemplated, including alternate forms ofpresentation of the symbols and sounds.

[0048] In the foregoing description, certain terms have been used forbrevity, clarity, and understanding, but no unnecessary limitations areto be implied therefrom beyond the requirements of the prior art,because such words are used for description purposes herein and areintended to be broadly construed. Moreover, the embodiments of theapparatus illustrated and described herein are by way of example, andthe scope of the invention is not limited to the exact details ofconstruction.

[0049] Having now described the invention, the construction, theoperation and use of preferred embodiment thereof, and the advantageousnew and useful results obtained thereby, the new and usefulconstructions, and reasonable mechanical equivalents thereof obvious tothose skilled in the art, are set forth in the appended claims.

What is claimed is:
 1. A method for evaluating an articulation disordercomprising the steps of: presenting to a user a symbol representative ofa word; prompting the user to pronounce the word represented by thesymbol into a microphone in signal communication with a processor;entering a phonetic representation of the user pronunciation into theprocessor; automatically determining whether an error exists in the userpronunciation; and if an error exists, automatically categorizing theerror.
 2. The method recited in claim 1, further comprising the step,following the prompting step, of displaying a frequency spectrum of theuser pronunciation.
 3. The method recited in claim 1, further comprisingthe step, following the prompting step, of broad casting a sample of acorrect pronunciation of the word.
 4. The method recited in claim 1,further comprising the step of issuing a report on an error in userpronunciation.
 5. The method recited in claim 1, further comprising thesteps of: saving the error in a database accessible by the processor;searching the database to determine whether a previous entry for theuser exists; and if a previous entry exists, comparing the error with anerror in the previous entry and determining a change with time.
 6. Themethod recited in claim 5, further comprising the step of issuing areport on the determined change.
 7. The method recited in claim 1,further comprising the step, if an error exists, of recommending atherapeutic program to address the error.
 8. The method recited in claim7, wherein the program recommending step comprises the steps of:searching a database comprising a plurality of records, each recordcomprising a picture and a word associated therewith; and automaticallygenerating a set of records from the plurality of records, each recordcontaining a word containing a problem speech sound representative ofthe error, the set of records for subsequent display and pronunciationby the user.
 9. The method recited in claim 1, wherein the presentingstep comprises displaying a picture on a display screen.
 10. The methodrecited in claim 1, wherein the error is selected from a groupconsisting of a substitution, a mispronunciation, and an omission. 11.The method recited in claim 1, wherein the determining step comprisesapplying a dialectical filter adapted to discriminate between an errorand a predetermined normal dialect word pronunciation.
 12. A method forevaluating an articulation disorder comprising the steps of: performinga pre-evaluation comprising the steps of: (a) presenting to a user asymbol representative of a word; (b) prompting the user to pronounce theword represented by the symbol into a microphone in signal communicationwith a processor; (c) entering a phonetic representation of the userpronunciation into the processor; (d) automatically determining whetheran error exists in the user pronunciation; and (e) if an error exists,automatically categorizing the error; repeating steps (a)-(e) apredetermined number of times; automatically generating a set ofsymbols, each symbol representative of a word containing at least one ofthe errors determined in the pre-evaluation; and performing anevaluation comprising performing steps (a)-(e) using the generated setof symbols.
 13. The method recited in claim 12, further comprisingautomatically generating a report summarizing the errors detected in theevaluation performing step.
 14. The method recited in claim 12, whereinthe evaluation performing step comprises automatically recognizing anunderlying commonality in the errors to achieve a diagnosis of a problemspeech sound.
 15. The method recited in claim 14, further comprising thestep of recommending a therapeutic program to address the diagnosedproblem speech sound.
 16. A method for evaluating an articulationdisorder comprising the steps of: (a) presenting to a user a symbolrepresentative of a word; (b) prompting the user to pronounce the wordrepresented by the symbol into a microphone in signal communication witha processor; (c) entering a phonetic representation of the userpronunciation into the processor; (d) automatically determining whetheran error exists in the user pronunciation; (e) if an error exists,automatically categorizing the error; repeating steps (a)-(e) apredetermined number of times; and correlating the categorized errors todetermine an existence of an articulation disorder.
 17. A method forevaluating an articulation disorder comprising the steps of: presentingto a user a motion picture representative of an action; prompting theuser to provide a narration on the action into a microphone in signalcommunication with a processor; entering a phonetic representation ofthe user pronunciation of the narration into the processor;automatically determining whether an error exists in the userpronunciation; and if an error exists, automatically categorizing theerror.
 18. A system for evaluating an articulation disorder comprising:a processor: an output device and an input device, each in signalcommunication with the processor; software means installable on theprocessor adapted to: present on the output device a symbolrepresentative of a word; prompt a user via the output device topronounce the word represented by the symbol; receive via the inputdevice a phonetic representation of the user pronunciation;automatically determine whether an error exists in the userpronunciation; and if an error exists, automatically categorize theerror.
 19. The system recited in claim 18, wherein the display devicecomprises at least one of a printer and a display screen and the inputdevice comprises at least one of a keyboard, a pointing device, and amicrophone.
 20. The system recited in claim 18, wherein the softwaremeans is further adapted to display on the display device a frequencyspectrum of the user pronunciation.
 21. The system recited in claim 18,further comprising broadcasting means in signal communication with theprocessor and wherein the software means is further adapted to direct asample of a correct pronunciation of the word to be broadcast via thebroadcast means.
 22. The system recited in claim 21, wherein thebroadcasting means comprises an audio speaker.
 23. The system recited inclaim 18, wherein the software means is further adapted to issue areport on an error in user pronunciation via the display device.
 24. Thesystem recited in claim 18, wherein the software means is furtheradapted, if an error exists, to recommend a therapeutic program toaddress the error.
 25. The system recited in claim 24, furthercomprising a database resident on the processor comprising a pluralityof records, each record comprising a picture and a word associatedtherewith; and wherein the software means is further adapted toautomatically generate a set of records from the plurality of records,each record containing a word containing a problem speech soundrepresentative of the error, the set of records for subsequent displayand pronunciation by the user.
 26. The system recited in claim 18,wherein the symbol comprises a picture and the output device comprises adisplay screen.
 27. The system recited in claim 18, wherein the outputdevice comprises a display screen, and wherein the software is adaptedto direct a presentation of a motion picture representative of an actionon the display screen.